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ABSTRACT 

The authors contend ttat it -Is necessary to assess 
the training needs of professional's who are routinely involved in 
evaluating exceptional children. They suggest that assessment 
personnel are not performing in the most ptbflclent vay posslt>le. The 
roles of the primary care physician in early identification and ot 
psychologists and special educators are examined. Issues in the 
conduct of needs assessment research, including political and 
methodological factors, are considered, seven recommendations for 
improving the performance o^ assessment personnel are listed, 
including more emphasis on delineating specific knowledge, skill, and 
personality components in assessment: cooperation between 
professionals and needs assessment researchers: and possible periodic 
reXicensing of personnel involved in assessment. (CL) 
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Abstract 

Ihis paper presents the argument that It Is necessary to assess the 
training needs of professionals ^o are routinely involved in the evalu" 
ation of execeptional children* -Ihe need Cor such assessment is based 
^upon_the_premi^e^^at at present, assessment personnel are not performing 
in the most proficient way possible* Support for this premise is pre* 
sented through discussion of the role of the primary care physician in early 
id^entification, and through discussion of the performance of evaluation duties 
by psychologists and special educators* Ihe paper also discusses a number of 
complex issues, including difficulties in the underlying concepts and method* 
ology of needs assessment, that must be addressed if the training needs of 
assessment personnel are to be fully identified* Finally, recommendations for 
improving the performance of assessment personnel are given* 
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Assessing the Assessors: A Necessary ^ 
and Important Training Function 
Thie paper will discuss the need Cor assessment of personnel involved 
in the evaluation of exceptional children* Those involved in evalua^tion 
of exceptional children help to provide the information upon which identifica- 
tioHi classification! placementi and educational programming are b^sed* 
Assessment professionals include hut are not limited to pediatricians or other 
primary care physicians, public health nuraes, social worlters, clinical or 

4- 

school psychologists, educational diagnosticians, and to an increasing extent, 
special education classroom teachers* 

There are a number of reasons for which the evaluation of Icnowledge, 
sl£|llS|^ and performance of professionals ^s typically undertalcen* These 
reasons include^ the gathering of information for decisions related to pre 
and inservice training, licensure and certification, productivity, and 

hiring and dismissal* ,This paper vill focus upon assessment of groups 

r 

of professionals for the purpose of identifying training needs* The paper 

vill, attempt tp show (a) vhy it is important to assess the training needs of 

+ 

assessment professionals, (b) what issues must be confronted if such efforts 

+ 

are to fully succeed, and (c) what can be done to improve the performance of 
these individuals* 

The Need for Assessing Professionals' Yrriaing Needs 
The basic premise underlying evaluation of professionals' assessment 
training needs is that ^present levels of perfonnarce ar± laclcing* Data 
exist to support the contention that the performance of those responsible 
for assessment of children is in fact deficient* The need for improved 
assessment performance is most clearly demonstrable ttyv physicians 
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providing prinary care to children^ and for psychologists and special educa'cors 
involved in the psychological and educational testing of exceptional children. 
The PrioMtry Care Physician and Assessment 



The primary care physician is thought to be the professional yith 
the best opportunity for early discovery of developmental dysfunction in 
children (Jacobs & Halker^ 1978; Masland^ 1969; Pearson^ 1968» 1976). As 
part of the routine pediatric examination^ this professional can gain early 
insight into whether a child is developing normally. Review of the familial^ 
maternal^ and perinatal history^"^ assessment of developmental status^ and 
evaluation of the parent-^child interaction by the physician, can all contribute 
to the early recognition of handicap, and the prompt referral of the infant or 
young child to the appropriate evaluation and intervention services. Those 
physicians itt the best position to function as early identifiers are p^diatri* 
cians^ family practitioners^ and general practitioners, as these professionals 
provide the major portion of primary medical care to the nation's children 
(Rosenbloom & Ongley, 1974; Task Force on Pediatric Education, 1978). 

At present, however, there is great concern that physicians are not 
making early referral to specialists of infants and young children exhibiting 
evidence of developmental disorder (Gofaan, 1969; Masland, 1969; Ruben, 1978). 
Tardiness in referral may result in the delayed provision of services, the 
persistence of unrealistic expectations by parents for the child's development, 
and the emergence of difficulties secondary to the developmental problem. 
Data to support the charge of delayed referral are limited, but Ho exist 
(e.g., Aldrich, Holliday, Colwell, Johnson, Smith, & Sharpley, Note 1: de la 
Cruz, 1976; Haynes, 1976; Kelly & Menolascino, 1975; Luterman & Chasin, 1970; 
Ruben & Rozycki, 1970; Shah. ChandUr. & Dale, 1978). 
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A number of explanations have been suggested fot the delayed referral 
of .children by physicians* One such suggested cause is a tendency on the 
part of the physician to postpone diagnosis^'^either in the hope that time 
will allow the child to *'catch^up** or» alternatively^ show the child to 
possess a clearly identifiable diagnostic entity (Tjossem^ 1976). Also 
suggested as a cause for tardy referral has been a lacic of Icnowledge of 
liistorical high risic signs (Pearson^ 1968) » and a lacic of knowledge of 
normal child development or of the measures used in its assessment 
(Lciwis^ tlote 2). Finally^ the problem may be related to a lacic of knowledge 
of available community evaluation and intervention services (Masland^ 
1969; Tjossem, 1976)* 

The role of needs assessment research in the problem of delayed referral 
should be clear* The task is first to discover^ through evaluation of personnel^ 
the specific content (or attitudinal) areas in which additional or restructured 
training is needed^ and then to provide that training* While a number of 
efforts have been made along these lines (e*g*» Becker^ 1978; Cornely^ Bachar, 
& Hankerd^ Note 3; Dworkin^ Shonkoff^ Leviton^ & Levine^ Note 4; Shah et al*» 
1978; Shonkoff^ Dworkin^ Leviton^ & Levine^ Note 5; Task Force on Pediatric 
Education^ 1978) » these attempts are not completely satisfactory* Most 
significant is the fact that the majority of these studies have failed to tap 
in any direct and comprehensive way» the knowledge base of physicians* Some 
studies (e*g*» Cornely et al*» Note 3; Tssk Force on Pediatric Education^ 1978) 
have attempted to assess training needs indirectly through opinion survey of 
physicians^ while others (e*g*» Shah et al*» 1978) have based their conclusions 
on parents' perceptions of the adeqtfacy with which the referral needs of 
their handicapped children were met by physicians* While both these types of 
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research are importanti the validity of basing pre and inservice curricular 
decisions on opinions and perceptions) rather than on direct assessment of 
knowle<tge| must be questioned* 

A second reason Cor questioning the value of these studies as needs 
assessments is that vith the exception of Becker (1978)» all studies mentioned 
have dealt vith pediatriciansi and pediatricians only* This situation is 
problematic since a substantial portion of the primsry medical care given 
U*S* children is provided by physicians other than the pediatrician (Rosenbloom 
Ongleyi 1974; Task Force on Pediatric Education^ 1978)* 

In sum» the training needs of primary care physicians vis-a-vis their 
ability to rapidly identify and refer special needs children have yet to be 
adequately assessed* 

The Psychologist, the Special Educator, and Assessment 

with regard to the educational and psychological evaluation of excep*^ 
tional children, problems in professional performance are also said to 
fixist (Bransford, 1974; Kirk & Kirk, 1978; Kirp & Kirp, 1976; MacMillan & 
Meyers, 1977; McBaniels, 1979; Meyers, Sundsti^om, & Yoshida, 1974; Salvia 
& Ysseldyke, 1978)* The general consensus Is that many of the psychologists 
and special educators involved in the evaluation of children lack the 
knowledge required for proper implementation of the assessment process* 
Some authors have even gone so far as to charge outright professional 
incompetence (e*g*, Kirp & Kirp, 1976)* 

The data to support the need for improved perfomance in educational 
and psychological evaluation of children are also limited, but growing* 
For example, a number of researchers have found large clerical error rates for 
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the scoring of intelligence test protocols byexaminers (Miller & Chansky^ 
1972; Miller^ Chansky^ & Gredler^ 1970; Warren & Brovn^ 1973). Others have , 
found tests being used in a routinized and' stereotyped manner with little^ 
regard Cor providing tedchers with meaningful instructioni»l feedback (e.g.^ 
Keogh» Kukic^ Becker^ McLoughlin^ . & Kukic^ 1975). Still other evidence has 
been provided to suggest deficits in professional performance as cause for the 
misclassificatian of children (e.g., Diana v. State Board of Education , 1970; 
Garrison & Hammill, 1971), 

While much of this research has focused upon the performance of clinical 
and school psychologists, there is rOas^n to believe that the assessment 
training needs of special educators are at least as pronounced as those of 
other assessment professionals. For example, Hammill and Blumberg (1967) and 
McLaughlip, Hinojosa^y and Trlica (1973) have found special educators to be 
deficient in understanding basic statistical symbols and temindlogy. Because 
a firm grounding in basic statistics is necessary for. proper understanding of 
the rudiments of educational and psychological measurement (Salvia & Ysseldyke 
1978), these findings suggest that special educators may encounter diffi&plty 
with many of the basic concepts necessary for proper test selection, adminis- 
tration, and interpretation. Bennett (1980) has presented data in support 
of this suggestion. His data indicate that educational diagnosticians do not 
possess an adequate level of proficiency in understanding or applying such 
basic concepts as reliability and validity in interpretation of test results* 
This lack of proficiency could well result in the misinterpretation of test 
results and hence, in the misclassif ication of children'* 

In conclusion, the existing literature concerning the performance of 
professionals involved in the assessment of except'ional children strongly 

8 



Assessing the Assessors 
7 

suggests thaj: additional attempts to specifically define the training needs of 
these professionals are badly needed. 

Issues in the Conduct of Heeds Assessment Research^ 

The Rules and Regulations of PL 94-142 (U*S* Office of Education,^ 
1977)^ provide a potential mechanism for the identification of the train- 
ing needs of assessment personnel* The regulations require states to determine 
annually the extent to ^ich qualified special education and related service 
personnel exist and the areas in .^ich those personnel need training* However, 
for this mandate to result in the identification of the specific training 
needs of assessment personnel, a number of complex issues must be confronted* 
These issues can be separated into those that are primarily of a political 
nature and those that are more related to the underlying concepts and method- 
ology of needs assessment research* 
Political Issues 

Under the rubric of political issues are three related problems* 

« 

These are (a) a lack of recognition^ of the existence of problems in the 
peiformance of assessment professionals, (b) a hesitancy on the part of 
professionals to cooperate with needs assessment efforts, and (c) a lack 
of financial support for needs assessment research* 

The lack of recognition within the fields of special education, psychology, 
and medicine of problems in professionals* assessment performance has no doubt 
contributed much to the paucity of needs assessment research with evaluation 
personnel* This lack of recognition has been aided in large part by a strong 
professional focus on the inadequacy of present testing instruments* Many 
available tests have been {severely criticized because they are said to be 
technically inadequate (Salvia & Ysseldyke, 1978), culturally hissed 
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(Ho1>1>s^ 1975,)^ or inappropriate as aids in the developicent of instructional 
programs (Wallace & Larsen^ 1978)* While criticisms of instrumentation are in 
many cases justified, they have been seized upon with such *'l>andwagon" fervor 
that attempts to bring to light other difficulties in assessment have been 
obscured* Ihough new emphasis is being given the identification of professional 
performance difficulties through the needs assessment provision of PL 94-142, 
problems in professional performance have yet to achieve status as a major 
concern in the assescment of exceptional children* 

A second political issue in the conduct of needs assessment research 
is that of professional cooperation* It has been the experience of these 
authors that professionals are sometimes hesitant to pa;.ticipate in needs 
assessment efforts* Ihis hesitancy is understandable* In part, it stems 
from a lack of awareness of the importance of such research* It may also 
result from the fact that profes.'^ionals perceive needs assessment research 
as an attempt to evaluate competence* The research may therefore be viewed as 
a threat to the well-being of those under study* Xt should be stressed, 
however, that needs assessment efforts are by definition undertaken for 
constructive purposes* They are jaitited at improving the performance of profes- 
sionals and the quality of service they provide* 

The final political issue is one of funds* Relative to other research 
and development areas in special education, support fc^r needs assessment 
efforts has been minimal* PL 94-142, for example, requires states to undertake 
annual needs assessment efforts but doos not earmark specific funds for this 
purpose* However, with an increase in awareness of the need for such research, 
more adequate support may be forthcoming* 
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Conceptual*Methodo logical Ijsues 

A number of complex concep'tual*methodological issues are involved in 
the conduct of needs assessment research. While comprehensive treatment 
of these issues is beyond the purposes of this paper, socte discussion of 
than will provide an introduction to the types of conceptual*methodologicaI 
problems encountered in: needs assessment. 

To begin with, there is the problem of what to b^b^bb. This problem 
may best be approached by first developing a fairly coaprehens ive ^under- 
standing of the nature of the job in question (Thorndike, 1949). At the 
least/ this involves a review of' previous research, reading of training 
manuals and other documentary materials, discussion of the job with personnel 
and personnel-trainers, and observation of personnel, performing the job. A 
more complete understanding of the job can be developed through actual 
experience in learning and performing some aspect of the job for a brief 
period. The results of these efforts should be fashioned into a compre- 
hensive and specific description of the job. The description should 
include a listing of the activities that make up the job and their relative 
importance, the conditions under which the job is performed, and the 
materials and equipment necessary for performance of the job. 

Once a description of the job has been formulated, the researcher 
can next set about the task of postulating which knowledge and skill 
domains, and personality traits, are~£^)licated in performance of each of 
the specific activities that compose the job. The domains and traits that 
are postulated as important are a set of hypotheses or inferences drawn 
from the jdb description. Due to practical necessity, the process of 
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linking traits and domaint with job activities is most often inferential 

rather than eapirical* Uofortunately^ there will be cases where logical 
+ ■ 

analyses will have omitted important componeotSi or specified a& important 
traits or domains that are in reality u;irelated to job performance* 

Given that there is scMue indication of performance deficits from previous 
X'oiearch or from discussion with experts^ the needs assessment researcher must 
next choose^ for study the knowledge^ skilly and personality componeots most 
probably responsible for the reported deficit* The most likely components for 
assessment will be those associated with the activities in which deficient 
performance has been reported* It goes almost without saying^ hoveveri that 
tu^re are numerous knowledge) skilli and personality requirements associated 
with any given professional activity* The choice of which one or ones to 

assess -is no simple matteri but should involve careful review of previous 

* 

research) consultation with expertSi and consideration of the relative impor- 

* J 

tance of each component to the activity in question* 

Having dealt with what to asB^BB^ the researcher must next confront 
the problem of how to assess it* Methods for evaluating a group's knowledge 
or skill in an area are many and diverse* ^.tAhey range from indirect (and 
somewhat'questionable) mean's like soliciting opinioosi to more direct means 
such as product examination (e*g*» review of case reports) and proficiency 
testing* Such factors as knowledge of specific job-relevant content areas 
(e*g*| for the school psychologist) knowledge of the^uses of particular 
achievement tests) can often be adequati^ly assessed through such direct 
measures as conventional paper and pencil testing* The measurement of more 
complex components of job performance however^ is a far more difficult 
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undertakings The more complex components of job performance (e.g^^ skill 
in establishing rapport) ntWy need to be evaluated through the use of 
simulations or through actual observation of job performance^ The problems 
of develbping valid^ reliable^ and cost-^eff icient measures of complex 
performance hove yet to be completely resolved^ In fact^ the situation 
is Such that Bennett (in press)^ in a comprehensive literature review^ found 
that well-researched irools for evaluating the performance of school psycho- 
logists were virtually non-existent* 

Once areas for assessment have been specified and measures selected 
or developed^ some means of interpreting the results must next be estab- 
lished* What level of performance on a proficiency test* for example, 
should be viewed as indicative of the existence of a training need?^ Ideally^ 
the level to be used is that which most accurately predicts deficient perform- 
ance on the job* Determination of this level would require a study designed 
to explore the relationship between performance on the needs assessment 
measure and performance on some more general measure of job performance (e.g*» 
a Supervisor's rating)* 'Unfortunately^ such additional studied, are in most^ 
cases impractical^ and in some cases impossible* In addition^ they raise a 
host of issues of their own (e*g*^ technical adequacy of the criterion measure). 
The interpretation of needs assessment results will^ then^ probably involve 
considerable dependence on the judgement of the researcher and other personnel 
experts in arriving at the minimal levels of knowledge and skill necessary for 
Successful performance of the job* 

In sum^ a number of complex issues must be dealt with , if full identifi- 
cation of professionals' assessment training needs is to be accomplished* These 

issues include questions of both political and methodological*conceptual concern. 

ft 
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^ The following recommendatipns are directed towards iinproving the 
competencies of professionals involved in the assessment of exceptional 

I 

children: . . x , . ^ ^ . v,.- . - ^ s 

, 1* The fields of special education, psychology, and medicine need 
tp become more aware of the problems that exist in professionals* perfor* 
mance pf the assessment function^ ' 

2* tfore efforts need to be directed at the delineation of specific 
knowledge, skill, and personality components central to performance of 
the evaluation function for the various groups of professionals engaged 

■J 

in assessments In addition, adequate tools for measurement of these knowledge, 
skill, and personality areas need to be developed along with methods for 
sensibly interpreting the results of performance on these measures^ 

3*. Identif icatfj-on of the training needs of assessment professionals 
through direct means using already existing and newly developed tools should 
be more widely and routinely undertaken^ Use of indirect measures as the so le 
means for determining training needs should be discouraged^ 

4s The results ^f training needs studies should be translated into 
curricula for pre and inservice training programs ^ These curricula need 
to be applied in the training of current and future assessment professionals^ 

5* Professionals need to be endouraged to cooperate with needs assessment 
efforts* This research will ultimately function to improve the competence of 
evaluation personnel and help them to more adequately serve exceptional 
children^ 

6s Funding agencies need to recognize the necessity to more fully 
support study in this areas 
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7* Finally* consideration ghould be given to requiring periodic relicensing 
of personnel involved in the assessment of exceptional children* "Kelicensing 
should be based in part upon objective evaluation of the professional's 
assessment- knowledge* Bkill^ and performance* ^ .t):.> 

Summary 

This paper has presented evidence to suggest that the |>erformEnce of 
professionals involved in the assessinent of exceptional children is deficient* 
In addition* the argument was advanced that the knowledge and skill neeos 
which underlie deficient performance have not yet been adequately identified* 
A number of complex issues that must be confronted if these training needs are 
to be fully determined were reviewed* Finally* recommendations for improving 
the performance of assessment personnel were given* 

i 

f 
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